(1) Field of the Invention
The invention relates to a method and an assembly for generating optical section images.
(2) Description of Related Art
In order to create three-dimensional images or mappings of objects, one often uses the technique of optical sectioning. A so-called optical section is an image, which contains information from a certain range of depth. Therefore, an optical system for the generation of optical section images performs selective imaging of those object details which are within the focal plane, while object details outside the focal plane are suppressed in the optical section image.
By means of recording of a series of optical section images located at different focal positions one can scan a three-dimensional object step by step. Thus a three-dimensional representation of an object or its topography can be analysed. In the following, the terms object and sample are used interchangeably. Especially in microscopy, the object investigated is often referred to as the sample.
One of the first methods for the generation of optical section images was the confocal microscope described in U.S. Pat. No. 3,013,467 entitled “Microscopy Apparatus”, which issued to Marvin Minsky in 1961 (hereinafter referred to as Reference No. 1). Here the imaging of details from outside the focal plane is suppressed by an arrangement of confocal pinholes.
Another approach for the generation of optical section images is structured illumination, as for example described in the article “Method of Obtaining Optical Sectioning by using Structured Light in a Conventional Microscope” by M. A. A. Neil, R. Ju{hacek over (s)}kaitis and T. Wilson, Optics Letters, Vol. 22, No. 24, p. 1905, 1997 (hereinafter referred to as Reference No. 2). Here, a structure, for example a grating, is projected into the sample to be imaged. This, in turn, creates a modulation of the light distribution within the sample. As is for example shown in Reference No. 2, the modulation depth has its largest value in the focal plane and marks the focal plane in that sense. In structured illumination the first step is to impose a modulation onto the illumination light, followed by a recording of different positions (phase steps) of the projected structure, where finally the optical section image is calculated from the recorded data.
For this purpose several arrangements were proposed. In U.S. Pat. No. 6,376,818, entitled “Microscopy Imaging Apparatus and Method”, which issued to Wilson, et al. in 2002 (hereinafter referred to as Reference No. 3), a grating is placed in a plane conjugated with the sample and moved perpendicular to the optical axis. In the different design described in U.S. Pat. No. 6,819,415, entitled “Assembly For Increasing the Depth Discrimination of an Optical Imaging System”, which issued to Gerstner, et al. in 2004, and in PCT Patent Publication No. WO02/12945, entitled “Assembly for Increasing the Depth Discrimination of an Optical Imaging System”, published in 2002 and naming Gerstner, et al. as inventors (hereinafter referred to as Reference No. 5), a parallel plate is inserted into the beam path and tilted, which laterally moves the illumination structure projected into the sample. For transparent or semi-transparent specimens a projection into the sample takes place while for non-transparent surface structures one refers to a projection onto the sample.
Another solution was proposed in the article “Real Time 3D Fluorescence Microscopy by Two-Beam Interference Illumination” by M. A. A. Neil, et al., Optics Communications, 153, 1998 (hereinafter referred to as Reference No. 6). Here, the illumination structure is created directly within the sample by means of interference.
The methods described in References Nos. 2-6 have the property that they require the recording of at least three individual images. All these methods have in common that artifacts and inaccuracies may arise during the positioning and projection of the structure since it requires fast and also accurate setting of a mechanical element. Details on artefacts and their compensation can for example be found in the article “Structured Illumination Microscopy: Artefact Analysis and Reduction Utilizing a Parameter Optimization approach” by Schaefer, Journal of Microscopy 216 (2), 165-174, 2004 (hereinafter referred to as Reference No. 7).
The different proposals for the implementation of the method differ with respect to the arrangement used to perform the change of the position of the illumination structure (phase setting). In particular, there are arrangements proposed without moving parts which, therefore, allow fixed alignment and a very good reproduction of the phase steps.
In U.S. Pat. No. 5,381,236, entitled “Optical Sensor for Imaging an Object”, which issued to Cohn G. Morgan (hereinafter referred to as Reference No. 11), an optical sensor for range finding of three-dimensional objects is described. Here, a periodic structure is projected onto the object where the illumination structure can be inverted, which corresponds to a phase shift of 180 degrees. The method proposed here is also based on a change of the illumination structure in two steps, but compared to Reference No. 11 it has the following differences:
In Reference No. 11, the individual elements of the illumination structure need to be exactly aligned with the individual elements of the detector (CCD pixels). This is a strong restriction for several reasons: the optical arrangement would need to be very accurate in terms of the magnification, to achieve matching of the illumination pattern with the detector. Furthermore, the alignment of the two structures in relation to each other would need to be very accurate and with sub-pixel precision. The probably most important limitation is due to the fact that even a small distortion in the optical image (e.g. a barrel or a pincushion distortion) makes it impossible to match the elements of the illumination pattern with those of the detector at a sub-pixel level for the whole field of view. The arrangement would require highly corrected and well adjusted optics, which would be an obstacle to widespread and robust applications.
For the proposed method, as opposed to Reference No. 11, the illumination pattern may be chosen freely with respect to type and pattern size because no exact matching of illumination pattern and detector is required. This allows adjusting the depth discrimination of the optical system, e.g. the thickness of the optical section generated (in a confocal microscope system this would correspond to the adjustment of the diameter of the confocal pinhole). With the present invention, this can be easily accomplished while it is impossible with the design in Reference No. 11 according to the state of the art. Therefore, characteristic disadvantages of Reference No. 11 can be avoided. An arrangement without moving parts can avoid disadvantages caused by position-inaccuracies and is also not disclosed in Reference No. 11.
At this stage, for a clear definition and separation from other, conceptually different methods for the measurement of surfaces based on structured illumination described in the literature are referenced. There are surface measurement methods based on triangulation in combination with structured illumination. As an example in the article “Shape Measurement by Use of Liquid-Crystal Display Fringe Projection with Two-Step Phase Shifting” by Chenggen Quan, et al., Applied Optics, Vol. 42, No. 13, 2003, 2329-2335 (hereinafter referred to as Reference No. 12) and further references cited in this document are mentioned. Triangulation evaluates the deformation of an illumination pattern (for example a fringe pattern) during projection onto an object, where the profile of the object is determined from the local phase of the projected pattern. The primary quantity measured is therefore the local phase. It is also characteristic, that projection of the pattern and detection are implemented using separate, non-coaxial optical systems or the object is tilted with respect to the optical axis.
For the present method for optical reproduction with depth discrimination, the goal is the separation (discrimination) of image signals, which originate from the focal plane, from those that correspond to the background. A depth profile of the sample can be obtained by means of axial scanning of the sample using a focussing device, where each of the partial images of the axial scan represents an optical section image. An optical section, sometimes also referred to as pseudo-confocal image, contains image signals from the focal plane only, while background signals are suppressed or removed using appropriate methods. A confocal microscope as described in Reference No. 1 does also produce optical sections; however, the task is accomplished by means of a different optical arrangement. For a method of structured illumination the depth discrimination is based on the detection of the local modulation as the primary quantity measured.
Furthermore, in the present invention, projection of the illumination pattern as well as the detection of the light from the sample is preferably performed through a single optic (objective), which is facing the sample. In contrast to that, triangulation works with projection and detection from different directions.
Another important aspect of implementations of the method of structured illumination with depth discrimination is the use of different wavelengths. Implementations currently known have problems when the wavelength is changed: due to remaining axial chromatic aberration, which may depend on the objective lens and the intermediate optics used, the structure projected (typically a mask) needs to be repositioned in axial direction. This requires relatively large movements within known microscope arrangements and therefore a lot of time for the movement of mechanical elements (see for example Reference No. 4).
The present invention proposes a novel arrangement, which does not require a mechanical axial movement of the projected mask structure anymore and therefore has advantages in terms of speed due to faster time-sequential or even time-simultaneous imaging with different wavelengths. In addition there are new arrangements proposed for the solution of the problem of chromatic correction, which use mechanical components but have lower complexity compared to the state of the art.
Another version of the principle of structured illumination according to the state of the art uses continuously moving illumination masks, which are projected into or onto the sample, are described in U.S. Pat. No. 6,687,052, entitled “Confocal Microscopy Apparatus and Method”, which issued to Wilson, et al. in 2004 (hereinafter referred to as Reference No. 8), and European Patent Publication No. EP1420281, entitled “Method and Apparatus for Optical Scanning With a Large Depth of Field”, which issued to Ralf Wolleschensky in 2004 (hereinafter referred to as Reference No. 9). Here a moving mask is used for encoding of the illumination structure as well as decoding. For that process it is characteristic, that the light originating from the sample passes the mask. In Reference No. 8, there was an arrangement described which is appropriate for use in a wide-field microscope. The arrangement in Reference No. 9 is predominantly useful in combination with a line scanner. Both methods described in Reference Nos. 8 and 9 have in common that two different signals are integrated on a spatially resolving detector, where the desired optical section image results from a simple subtraction of both image datasets. The arrangements in Reference Nos. 8 and 9 have the disadvantage in common that the light to be detected originating from the sample is attenuated by the mask before it is registered by the detector. This is relevant in particular when weak light signals are observed, which occurs especially in fluorescence microscopy.
In the article “A Wide-Field-Time-Domain Fluorescence Lifetime Imaging Microscope with Optical Sectioning” by S. E. D. Webb, et al., Review of Scientific Instruments, Vol. 73, No. 4, 2002, 1898-1907 (hereinafter referred to as Reference No. 13), it was shown how the method of structured illumination can be combined with the method of fluorescence lifetime measurement. Here the method from Reference No. 2 and Reference No. 3 according to the state of the art is used, which requires the recording of a sequence of three illumination steps. Measurement of fluorescence lifetime is implemented using a combination of a detector with very high temporal resolution (gated optical intensifier) and pulsed laser excitation. A different version for the determination of fluorescence lifetime is “frequency domain FLIM”, where the excitation light is modulated periodically and the lifetime is determined from the phase shift of the detected signal with respect to excitation signal. The present invention does also allow measuring the fluorescence lifetime in optical section images with the recording of only two single images per optical section calculated, if appropriate detectors and light sources are used.
The article “Single-Exposure Optical Sectioning by Color Structured Illumination Microscopy” by L. G. Krzewina and M. K. Kim Optics Letters, Vol. 31, No. 4, 2006, 477-479 (hereinafter referred to as Reference No. 14) covers a method for the generation of optical section images from only one image. Here the method using the illumination patterns is implemented as presented in Reference Nos. 2 and 3. During the process, the three illumination patterns projected into or onto the sample and the phase steps from the sample are encoded using light of different wavelengths. This allows time-simultaneous projection and detection. This technique offers advantages in terms of speed, but also creates other problems. The use of fluorescence microscopy is not feasible here since the dyes have pre-determined spectral properties (excitation wavelength and emission wavelength). Furthermore, problems occur when the sample exhibits inhomogeneous spectral properties in reflection mode. Reference No. 14 is therefore considered a special case of the method in Reference No. 2 and Reference No. 3, where limitations occur due to the wavelength encoding of the illumination steps. For the present invention, such limitations are not present since one does not need spectral encoding of the illumination steps.